…nflating-metrics-background-work

Two general-purpose utilities used by the client-side stats aggregator
work (PR #11382 and follow-ups), extracted into their own change so the
metrics-specific PRs can build on a smaller, reviewable foundation.

  - Hashtable: a generic open-addressed-ish bucket table abstraction
    keyed by a 64-bit hash, with a public abstract Entry type so client
    code can subclass it for higher-arity keys. The metrics aggregator
    uses it to back its AggregateTable.

  - LongHashingUtils: chained 64-bit hash combiners with primitive
    overloads (boolean, short, int, long, Object). Used in place of
    varargs combiners to avoid Object[] allocation and boxing on the
    hot path.

No callers within internal-api itself yet -- the metrics aggregator PR
will introduce the first usages.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Standalone classes for swapping the consumer-side LRUCache<MetricKey,
AggregateMetric> with a multi-key Hashtable in the next commit. No call sites
use them yet.

- AggregateEntry extends Hashtable.Entry, holds the canonical MetricKey, the
  mutable AggregateMetric, and copies of the 13 raw SpanSnapshot fields for
  matches(). The 64-bit lookup hash is computed via chained
  LongHashingUtils.addToHash calls (no varargs, no boxing of short/boolean).
- AggregateTable wraps a Hashtable.Entry[] from Hashtable.Support.create.
  findOrInsert(SpanSnapshot) walks the bucket comparing raw fields, falling
  back to MetricKeys.fromSnapshot on a true miss. On cap overrun, it scans
  for an entry with hitCount==0 and unlinks it; if none, it returns null and
  the caller drops the data point.
- MetricKeys.fromSnapshot extracts the canonicalization logic (DDCache
  lookups + UTF8 encoding) from Aggregator.buildMetricKey, so the helper can
  be called from AggregateTable on miss.

This also commits Hashtable and LongHashingUtils (added earlier, previously
uncommitted) and lifts Hashtable.Entry / Hashtable.Support visibility so
client code outside datadog.trace.util can build higher-arity tables -- the
case the javadoc describes but the original visibility didn't actually
support. Specifically: Entry is now public abstract with a protected ctor;
keyHash, next(), and setNext() are public; Support's create / clear /
bucketIndex / bucketIterator / mutatingBucketIterator methods are public.

Tests: AggregateTableTest covers hit, miss, distinct-by-spanKind, peer-tag
identity (including null vs non-null), cap overrun with stale victim, cap
overrun with no victim (returns null), expungeStaleAggregates, forEach,
clear, and that the canonical MetricKey is built at insert.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Replace LRUCache<MetricKey, AggregateMetric> with the AggregateTable added
in the prior commit. The hot path in Drainer.accept becomes:

  AggregateMetric aggregate = aggregates.findOrInsert(snapshot);
  if (aggregate != null) {
      aggregate.recordOneDuration(snapshot.tagAndDuration);
      dirty = true;
  } else {
      healthMetrics.onStatsAggregateDropped();
  }

On the steady-state hit path the lookup is a 64-bit hash compute + bucket
walk + matches(snapshot) -- no MetricKey allocation, no SERVICE_NAMES /
SPAN_KINDS / PEER_TAGS_CACHE lookups. The canonical MetricKey is now built
once per unique key at insert time, in MetricKeys.fromSnapshot.

Behavioral change in the cap-overrun path
-----------------------------------------

The old LRUCache evicted least-recently-used: at cap, a new insert would
push out the oldest entry regardless of whether it was live or stale.
AggregateTable instead scans for a hitCount==0 entry to recycle, and drops
the new key if none exists. Practical impact: in the common case where
the table holds a stable set of recurring keys, an unrelated burst of new
keys is dropped (and reported via onStatsAggregateDropped) rather than
evicting the established keys. The existing test that asserted "service0
evicted in favor of service10" is updated to assert the new semantics.
The other cap-related test ("should not report dropped aggregate when
evicted entry was already flushed") still passes unchanged: after report()
clears all entries to hitCount=0, the next wave of inserts recycles them.

Threading fix
-------------

ConflatingMetricsAggregator.disable() used to call aggregator.clearAggregates()
and inbox.clear() directly from the Sink's IO event thread, racing with the
aggregator thread mid-write. The race was tolerable for LinkedHashMap; it
is not for AggregateTable (chain corruption can NPE or loop). disable()
now offers a ClearSignal to the inbox so the aggregator thread itself
performs the table clear and the inbox.clear(). Adds one SignalItem
subclass + one branch in Drainer.accept; preserves the single-writer
invariant for AggregateTable end-to-end.

Removed: LRUCache import, AggregateExpiry inner class, the static
buildMetricKey / materializePeerTags / encodePeerTag helpers (now in
MetricKeys).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

MetricKey existed for two reasons -- the prior LRUCache key role (now handled
by AggregateTable's Hashtable.Entry mechanics) and as the labels argument
to MetricWriter.add. The first is gone; the second is the only thing keeping
MetricKey alive. Fold its UTF8-encoded label fields onto AggregateEntry,
change MetricWriter.add to take AggregateEntry directly, and delete
MetricKey + MetricKeys.

What AggregateEntry now holds
-----------------------------

- 10 UTF8BytesString label fields (resource, service, operationName,
  serviceSource, type, spanKind, httpMethod, httpEndpoint, grpcStatusCode,
  and a List<UTF8BytesString> peerTags for serialization).
- 3 primitives (httpStatusCode, synthetic, traceRoot).
- AggregateMetric (the value being accumulated).
- The raw String[] peerTagPairs is retained alongside the encoded peerTags
  -- matches() compares it positionally against the snapshot's pairs; the
  encoded form is only consumed by the writer.

matches(SpanSnapshot) compares the entry's UTF8 forms to the snapshot's raw
String / CharSequence fields via content-equality (UTF8BytesString.toString()
returns the underlying String in O(1)). This closes a latent bug in the
prior raw-vs-raw matches(): if one snapshot delivered a tag value as String
and a later snapshot delivered the same content as UTF8BytesString, the old
Objects.equals would return false and the table would split into two
entries. Content-equality matching collapses them into one.

Consolidated caches
-------------------

The static UTF8 caches that used to live partly on MetricKey (RESOURCE_CACHE,
OPERATION_CACHE, SERVICE_SOURCE_CACHE, TYPE_CACHE, KIND_CACHE,
HTTP_METHOD_CACHE, HTTP_ENDPOINT_CACHE, GRPC_STATUS_CODE_CACHE, SERVICE_CACHE)
and partly on ConflatingMetricsAggregator (SERVICE_NAMES, SPAN_KINDS,
PEER_TAGS_CACHE) are all now on AggregateEntry. The split was duplicating
work -- SERVICE_NAMES and SERVICE_CACHE both cached service-name to
UTF8BytesString. One cache per field now.

API change: MetricWriter.add
----------------------------

Was: add(MetricKey key, AggregateMetric aggregate)
Now: add(AggregateEntry entry)

The aggregate lives on the entry. Single-arg.

SerializingMetricWriter reads the same UTF8 fields off AggregateEntry that it
previously read off MetricKey; the wire format is byte-identical.

Test impact
-----------

AggregateEntry.of(...) takes the same 13 positional args new MetricKey(...)
took, so test diffs are mostly mechanical:
  new MetricKey(args) -> AggregateEntry.of(args)
  writer.add(key, _)  -> writer.add(entry)

ValidatingSink in SerializingMetricWriterTest now iterates List<AggregateEntry>
directly. ConflatingMetricAggregatorTest's Spock matchers (~36 sites) rely
on AggregateEntry.equals comparing the 13 label fields (not the aggregate)
so the mock matches by labels regardless of the aggregate state at call time;
post-invocation closures verify aggregate state.

Benchmarks (2 forks x 5 iter x 15s)
-----------------------------------

The change is consumer-thread only; producer publish() is unchanged.

  SimpleSpan bench:   3.123 +- 0.025 us/op   (prior: 3.119 +- 0.018)
  DDSpan bench:       2.412 +- 0.022 us/op   (prior: 2.463 +- 0.041)

Both within noise -- the win is structural (one less class, one less
allocation per miss, one fewer cache layer) rather than benchmarked.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The prior commit ran every snapshot through the cardinality handlers but still
hashed the raw snapshot fields. When a field exceeded its cardinality budget
the handlers collapsed many distinct values to a single "blocked_by_tracer"
sentinel, but the raw hashes were still all different -- so the blocked entries
fragmented across the AggregateTable. This commit makes hash + match work off
the canonical (post-handler) UTF8BytesString fields, so blocked values land in
the same bucket and merge into one entry.

How the lookup path changes
---------------------------

A new package-private AggregateEntry.Canonical scratch buffer:

  - holds the 10 canonical UTF8BytesString refs, primitives, peerTags list,
    and the precomputed keyHash;
  - exposes populate(SpanSnapshot) which runs each field through the
    appropriate handler and computes the long hash from the canonical refs;
  - exposes matches(AggregateEntry) for content-equality lookup;
  - exposes toEntry(AggregateMetric) which copies its refs into a fresh
    AggregateEntry on miss.

AggregateTable holds one Canonical instance and reuses it per findOrInsert.
On a hit nothing is allocated -- the buffer's refs feed the bucket walk and
matches() directly. On a miss the refs are copied into the new entry and the
buffer is overwritten on the next call.

Hash function
-------------

hashOf now takes UTF8BytesString fields (plus primitives + peerTags list)
instead of raw CharSequence/String from the snapshot. UTF8BytesString.hashCode
returns the underlying String's hash, so:

  - content-equal entries built via AggregateEntry.of(...) (test factory,
    bypasses handlers) produce the same hash as entries built via
    Canonical.toEntry(...) (production, via handlers);
  - all values that collapsed to "blocked_by_tracer" share that sentinel
    instance and therefore that hashCode -- they land in the same bucket and
    merge into one entry.

Matches
-------

The SpanSnapshot-keyed matches() on AggregateEntry is gone. Lookup goes
through Canonical.matches(entry) which compares the buffer's UTF8 fields
against the entry's UTF8 fields via Objects.equals (content equality on
UTF8BytesString). This is needed because across handler resets the
UTF8BytesString instance referenced by an existing entry differs from the
freshly-issued instance for the same content -- content-equality lets the
existing entry survive resets.

The peerTagPairsRaw field on AggregateEntry was previously kept for matching
against snapshot.peerTagPairs (the flat String[]). Canonical.matches uses
List.equals on the encoded UTF8 peerTags directly, so peerTagPairsRaw is
dropped.

New test in AggregateTableTest -- cardinalityBlockedValuesCollapseIntoOneEntry
inserts 50 distinct services into a table whose SERVICE_HANDLER has a
cardinality limit of 32, and asserts the final size is 33 (the 32 in-budget
services plus a single collapsed "blocked_by_tracer" entry, not 50 separate
entries).

Benchmarks (2 forks x 5 iter x 15s) -- producer side unchanged:

  SimpleSpan bench:   3.117 +- 0.026 us/op   (prior: 3.114 +- 0.045)
  DDSpan bench:       2.344 +- 0.114 us/op   (prior: 2.364 +- 0.113)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…chema-indexed handlers

Replaces the producer's early {@code (name, value)}-pair encoding with a
schema-based design: peer-tag values are captured into a parallel String
array, and the consumer applies the matching {@link TagCardinalityHandler}
by index using a {@link PeerTagSchema}'s parallel name/handler arrays.

This removes the {@code Map<String, TagCardinalityHandler>} the prior commit
left in {@code AggregateEntry} -- handler lookup is now a single array
dereference instead of a hashmap probe.

PeerTagSchema
-------------

New package-private class that holds:

  - {@code String[] names}    -- peer-tag names in stable order
  - {@code TagCardinalityHandler[] handlers} -- parallel to names

Two schemas exist: a static singleton {@code INTERNAL} for the internal-kind
{@code base.service} case, and a {@code CURRENT} schema for the peer-
aggregation kinds (client/producer/consumer) that lazily refreshes when
{@code features.peerTags()} returns a different set of names.

Each {@link SpanSnapshot} captures the schema reference it was built against
so producer and consumer agree on the indexing even if {@code CURRENT}
changes between capture and consumption.

A fast-path identity check (cached last input Set instance) keeps the
{@code currentSyncedTo} call cheap: when the producer hands in the same
Set instance as last time -- the steady-state case --
{@code currentSyncedTo} returns immediately without iterating names. The
{@code matches()} loop only runs when the Set instance changes, which in
production is rare (only on remote-config reconfiguration).

Snapshot shape
--------------

{@code SpanSnapshot.peerTagPairs} (a flat {@code [name0, value0, name1,
value1, ...]} array) is replaced by:

  - {@code PeerTagSchema peerTagSchema}  -- nullable; schema for the values
  - {@code String[] peerTagValues}       -- parallel to schema.names

The producer captures only values; the consumer constructs the encoded
{@code "name:value"} UTF8 forms via {@code schema.handler(i).register(value)}
on its own thread.

Consumer-side cleanups bundled in
---------------------------------

While here, also addresses the perf review items raised against the prior
commit:

  - {@code hashOf}'s peer-tag loop is now indexed iteration; no more
    iterator allocation per snapshot.
  - {@code Canonical} now owns a reusable {@code peerTagsBuffer} ArrayList
    that's cleared+refilled per {@code populate} call -- zero allocation
    on the hit path. The buffer is copied into an immutable list only on
    miss when the entry needs to own it long-term.
  - {@code Canonical.matches} uses indexed list comparison; no iterator
    alloc in {@code List.equals}.
  - The {@code HashMap<String, TagCardinalityHandler> PEER_TAG_HANDLERS}
    on {@code AggregateEntry} is gone, replaced by the {@link
    PeerTagSchema}'s parallel array layout.

Benchmark (2 forks x 5 iter x 15s)
----------------------------------

  SimpleSpan bench:   3.165 +- 0.032 us/op   (prior: 3.117 +- 0.026)
  DDSpan bench:       2.727 +- 0.018 us/op   (prior: 2.344 +- 0.114)

Some producer-side regression from the per-snapshot schema sync (volatile
read + identity check). The fast-path identity comparison keeps it small;
hoisting the sync out of the per-snapshot loop is possible but would change
behavior in the edge case where {@code features.peerTags()} returns
different Sets within a single trace (covered by an existing test). Choosing
correctness over the marginal speedup.

Tests
-----

AggregateTableTest's snapshot builder is updated to construct a schema +
values via {@code PeerTagSchema.currentSyncedTo}, exercising the same code
path as production. Existing peer-tag test in {@code
ConflatingMetricAggregatorTest} still passes unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The "Conflating" in the name dates from the prior design that used a Batch
pool + pending map to conflate up to 64 hits per inbox slot. That mechanism
is gone -- the producer now publishes one SpanSnapshot per span and the
consumer's AggregateTable is the conflation point. The new name matches the
existing protocol/metric terminology (HealthMetrics.onClientStat*,
stats.flush_payloads, etc.).

File renames:

  ConflatingMetricsAggregator.java         -> ClientStatsAggregator.java
  ConflatingMetricAggregatorTest.groovy    -> ClientStatsAggregatorTest.groovy
  ConflatingMetricsAggregatorBenchmark     -> ClientStatsAggregatorBenchmark
  ConflatingMetricsAggregatorDDSpan*       -> ClientStatsAggregatorDDSpan*

Plus all symbol references in MetricsAggregatorFactory and the test fixtures
that referenced the old class name.

No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Three small follow-ups carried over from a /techdebt pass:

- TracerHealthMetrics: previousCounts array was sized 51, but the prior
  commits added a 52nd reporter (statsInboxFull). Without this fix the new
  counter's report() call would throw ArrayIndexOutOfBoundsException; the
  Flush task swallows that exception, so the failure would be silent
  (statsInboxFull would just never make it to statsd).

- Aggregator: removes the now-dead public clearAggregates() method. The
  ClearSignal route from ClientStatsAggregator.disable() supplanted it
  several commits ago; the method had no remaining callers.

- TagCardinalityHandler: removes the unused register(TagMap.Entry) overload
  and its isValidType helper. The String-keyed overload covers all current
  callers (AggregateEntry's peer-tag canonicalization).

- PeerTagSchema: spotless-driven javadoc reflow only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

ClientStatsAggregator.publish was calling features.peerTags() +
PeerTagSchema.currentSyncedTo for every span. Peer-tag configuration is
stable for the duration of a single trace publish in production --
DDAgentFeaturesDiscovery returns the same Set instance until remote-config
reconfiguration -- so the per-snapshot sync is wasted work.

Move the sync to once per publish(trace) and pass the resolved schema to
the inner publish(span, isTopLevel, peerAggSchema). INTERNAL-kind spans
still use the static PeerTagSchema.INTERNAL regardless.

Behavior boundary
-----------------

Schema changes from features.peerTags() now take effect at the next
publish(trace) call rather than mid-trace. Production-equivalent (a trace
takes microseconds to milliseconds; remote-config refreshes are seconds
apart), but a Spock test that used `>>> [...]` to mock different peerTags()
returns on successive calls within one trace no longer makes sense in the
new model. That test is rewritten to assert the production-relevant case:
peer-tag NAMES are stable, peer-tag VALUES vary per span, distinct value
combinations produce distinct aggregate buckets.

Benchmark (2 forks x 5 iter x 15s)
----------------------------------

  SimpleSpan bench:   3.133 +- 0.057 us/op   (prior: 3.165 +- 0.032)
  DDSpan bench:       2.454 +- 0.082 us/op   (prior: 2.727 +- 0.018)

Recovers ~270 ns/op on the DDSpan bench -- most of the regression introduced
by the per-snapshot lookup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

JFR profiling showed ~21% of producer CPU time spent in tag-map lookups
during ClientStatsAggregator.publish. One of those lookups -- span.kind --
is redundant because DDSpanContext already caches the kind as a byte
ordinal that resolves to a String via a small array.

- Add CoreSpan.getSpanKindString() with a default that falls back to the
  tag map for non-DDSpan impls; DDSpan overrides to delegate to the
  context's cached resolution.
- Hoist schema.names array out of the capturePeerTagValues loop.
- Avoid an unnecessary toString() in isSynthetic by declaring
  SYNTHETICS_ORIGIN as String and using contentEquals.

Benchmark (ClientStatsAggregatorDDSpanBenchmark):
  before: 2.410 us/op
  after:  1.995 us/op  (~17% improvement)
  vs. master baseline (6.428 us/op): now ~3.2x faster.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Captures the producer/consumer split, the canonical-key trick that makes
cardinality-blocking actually save space, the once-per-trace peer-tag
schema sync, the role of each file in datadog.trace.common.metrics, and
the rationale behind the redesign from ConflatingMetricsAggregator.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Bring the new util/ files in line with google-java-format
(tabs → spaces, line wrapping, javadoc list markup) so
spotlessCheck passes in CI.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Compares Hashtable.D1 and Hashtable.D2 against equivalent HashMap
usage for add, update, and iterate operations. Each benchmark thread
owns its own map (Scope.Thread), but @threads(8) is used so the
allocation/GC pressure that Hashtable is designed to avoid surfaces
in the throughput numbers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

- Guard Support.sizeFor against overflow and use Integer.highestOneBit;
  reject capacities above 1 << 30 instead of looping forever.
- Add braces around single-statement while bodies in BucketIterator.
- Split HashtableBenchmark into HashtableD1Benchmark / HashtableD2Benchmark.
- Add regression tests for Support.sizeFor bounds.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The 5-arg Object overload was forwarding only obj0..obj3 to the int
overload, silently dropping obj4. Also align LongHashingUtils.hash 3-arg
signature with its 2/4/5-arg siblings (int parameters) and strengthen
the 5-arg HashingUtilsTest to detect the missing-arg regression.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

- Split D1Tests and D2Tests into HashtableD1Test and HashtableD2Test;
  extract shared test entry classes into HashtableTestEntries.
- Reduce visibility of LongHashingUtils.hash(int...) chaining overloads
  to package-private; they are internal building blocks.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

peerTagSchemaFor now takes the already-resolved spanKind String and calls
SpanKindFilter.matches(String) directly, avoiding two tag-map lookups that
getSpanKindString() had already paid for.

Canonical.peerTagsBuffer initialized to EMPTY_TAGS instead of null,
removing the null check in populate()'s hashOf() call and the null branch
in populatePeerTags().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ig to follow-up PR

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…mponent config lands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The factory was test-only but lived in the production class. Making the
constructor package-private lets AggregateTableTest (same package) call it
directly; the test-only method is removed from the production API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…-private constructor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…alls

onStatsAggregateDropped() now emits an immediate collapsed_spans count
(RFC §2) in addition to the existing periodic stats.dropped_aggregates
report. Update the test to use CountDownLatch(2) and verify both calls.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- AggregateTable: use Hashtable.Support.xxx() qualified form, fix stale
  ConflatingMetricsAggregator reference in Javadoc
- PropertyCardinalityHandler: add @VisibleForTesting, extract
  MAX_CARDINALITY_LIMIT constant, restructure inline comments in
  register() per review suggestions, add cycle definition to class doc

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The factory was always test-only and the class Javadoc on AggregateEntryTestUtils
already stated the intent: "Lives in src/test so the production class stays free
of test-only API." Moving it there completes that design.

To enable the move:
- AggregateEntry constructor: private → package-private
- AggregateEntry.createUtf8(): private static → package-private static
Both are non-public internal helpers; widening to package-private exposes them
only within the same source set.

AggregateEntryTestUtils.of() now owns the implementation directly.
All callers updated: AggregateEntryTest (Java) and ClientStatsAggregatorTest (Groovy).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…tyHandler

Remove DDCache history, storage layout details, and reset mechanics from
the class headers -- those belong in a design doc or the inline comments
that already explain them at the call site. Keep just what's non-obvious
from the class name: the cycle/budget concept and the prior-cycle reuse
invariant.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…alityHandler

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>